A Deep Learning Framework for Unsupervised Affine and Deformable Image Registration

Image registration, the process of aligning two or more images, is the core technique of many (semi-)automatic medical image analysis tasks. Recent studies have shown that deep learning methods, notably convolutional neural networks (ConvNets), can be used for image registration. Thus far training of ConvNets for registration was supervised using predefined example registrations. However, obtaining example registrations is not trivial. To circumvent the need for predefined examples, and thereby to increase convenience of training ConvNets for image registration, we propose the Deep Learning Image Registration (DLIR) framework for \textit{unsupervised} affine and deformable image registration. In the DLIR framework ConvNets are trained for image registration by exploiting image similarity analogous to conventional intensity-based image registration. After a ConvNet has been trained with the DLIR framework, it can be used to register pairs of unseen images in one shot. We propose flexible ConvNets designs for affine image registration and for deformable image registration. By stacking multiple of these ConvNets into a larger architecture, we are able to perform coarse-to-fine image registration. We show for registration of cardiac cine MRI and registration of chest CT that performance of the DLIR framework is comparable to conventional image registration while being several orders of magnitude faster.Comment: Accepted: Medical Image Analysis - Elsevie

Local Implicit Neural Representations for Multi-Sequence MRI Translation

In radiological practice, multi-sequence MRI is routinely acquired to characterize anatomy and tissue. However, due to the heterogeneity of imaging protocols and contra-indications to contrast agents, some MRI sequences, e.g. contrast-enhanced T1-weighted image (T1ce), may not be acquired. This creates difficulties for large-scale clinical studies for which heterogeneous datasets are aggregated. Modern deep learning techniques have demonstrated the capability of synthesizing missing sequences from existing sequences, through learning from an extensive multi-sequence MRI dataset. In this paper, we propose a novel MR image translation solution based on local implicit neural representations. We split the available MRI sequences into local patches and assign to each patch a local multi-layer perceptron (MLP) that represents a patch in the T1ce. The parameters of these local MLPs are generated by a hypernetwork based on image features. Experimental results and ablation studies on the BraTS challenge dataset showed that the local MLPs are critical for recovering fine image and tumor details, as they allow for local specialization that is highly important for accurate image translation. Compared to a classical pix2pix model, the proposed method demonstrated visual improvement and significantly improved quantitative scores (MSE 0.86 x 10^-3 vs. 1.02 x 10^-3 and SSIM 94.9 vs 94.3

Joint optimization of a β\beta-VAE for ECG task-specific feature extraction

Electrocardiography is the most common method to investigate the condition of the heart through the observation of cardiac rhythm and electrical activity, for both diagnosis and monitoring purposes. Analysis of electrocardiograms (ECGs) is commonly performed through the investigation of specific patterns, which are visually recognizable by trained physicians and are known to reflect cardiac (dis)function. In this work we study the use of $\beta$-variational autoencoders (VAEs) as an explainable feature extractor, and improve on its predictive capacities by jointly optimizing signal reconstruction and cardiac function prediction. The extracted features are then used for cardiac function prediction using logistic regression. The method is trained and tested on data from 7255 patients, who were treated for acute coronary syndrome at the Leiden University Medical Center between 2010 and 2021. The results show that our method significantly improved prediction and explainability compared to a vanilla $\beta$-VAE, while still yielding similar reconstruction performance.Comment: Conference paper, 10 pages, 3 figures, 1 tabl

Joint Intensity Inhomogeneity Correction for Whole-Body MR Data

Abstract. Whole-body MR receives increasing interest as potential alternative to many conventional diagnostic methods. Typical whole-body MR scans contain multiple data channels and are acquired in a multistation manner. Quantification of such data typically requires correction of two types of artefacts: different intensity scaling on each acquired image stack, and intensity inhomogeneity (bias) within each stack. In this work, we present an all-in-one method that is able to correct for both mentioned types of acquisition artefacts. The most important properties of our method are: 1) All the processing is performed jointly on all available data channels, which is necessary for preserving the relation between them, and 2) It allows easy incorporation of additional knowledge for estimation of the bias field. Performed validation on two types of whole-body MR data confirmed superior performance of our approach in comparison with state-of-the-art bias removal methods

Robust contour propagation using deep learning and image registration for online adaptive proton therapy of prostate cancer

Purpose To develop and validate a robust and accurate registration pipeline for automatic contour propagation for online adaptive Intensity‐Modulated Proton Therapy (IMPT) of prostate cancer using elastix software and deep learning. Methods A three‐dimensional (3D) Convolutional Neural Network was trained for automatic bladder segmentation of the computed tomography (CT) scans. The automatic bladder segmentation alongside the computed tomography (CT) scan is jointly optimized to add explicit knowledge about the underlying anatomy to the registration algorithm. We included three datasets from different institutes and CT manufacturers. The first was used for training and testing the ConvNet, where the second and the third were used for evaluation of the proposed pipeline. The system performance was quantified geometrically using the dice similarity coefficient (DSC), the mean surface distance (MSD), and the 95% Hausdorff distance (HD). The propagated contours were validated clinically through generating the associated IMPT plans and compare it with the IMPT plans based on the manual delineations. Propagated contours were considered clinically acceptable if their treatment plans met the dosimetric coverage constraints on the manual contours. Results The bladder segmentation network achieved a DSC of 88% and 82% on the test datasets. The proposed registration pipeline achieved a MSD of 1.29 ± 0.39, 1.48 ± 1.16, and 1.49 ± 0.44 mm for the prostate, seminal vesicles, and lymph nodes, respectively, on the second dataset and a MSD of 2.31 ± 1.92 and 1.76 ± 1.39 mm for the prostate and seminal vesicles on the third dataset. The automatically propagated contours met the dose coverage constraints in 86%, 91%, and 99% of the cases for the prostate, seminal vesicles, and lymph nodes, respectively. A Conservative Success Rate (CSR) of 80% was obtained, compared to 65% when only using intensity‐based registration. Conclusion The proposed registration pipeline obtained highly promising results for generating treatment plans adapted to the daily anatomy. With 80% of the automatically generated treatment plans directly usable without manual correction, a substantial improvement in system robustness was reached compared to a previous approach. The proposed method therefore facilitates more precise proton therapy of prostate cancer, potentially leading to fewer treatment‐related adverse side effects